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ABSTRACT 


A 


Image  coding  techniques  which  have  proven  to  be  useful 
for  transmission  purposes  have  seldom  been  applied  to  image 
analysis.  This  report  examines  the  application  of  one  such 
technique,  run-length  encoding,  to  problems  in  the  scene 
analysis  domain.  Several  operations  within  a run-length 
encoded  data  structure  are  discussed,  including  area  access, 
region  growing,  edge  following,  and  corner  identification. 

\ 

I . Run  Length  Coding  of  Images 
A.  Introduction 

The  basic  data  structure  used  to  represent  M x N images 
is  a one-dimensional  list  of  intensity  values  of  length  MN. 

This  structure  is  necessary  because  almost  all  computers 
are  incapable  of  any  sort  of  an  explicit  operation  on  the 
most  natural  representation  of  an  image,  an  M x N array  of 
intensities.  There  are  a few  exceptions,  of  course,  including 
the  CLIP  series  of  parallel  array  processors  [1,2]. 

For  a regular  array  (uniform  length  of  rows  and  columns) 
indexing  within  the  list  is  fairly  easy.  For  instance,  a 
doubly  subscripted  Fortran  variable,  I(X,Y)  Is  the  ((Y-1)*N)) 
+X-1  member  of  the  list  I representing  an  M x N array 
(1  < X < M) , (1  < Y < N) . 

For  those  arrays  which  do  not  contain  uniformly 
dimensioned  rows  or  columns  (and.  Including  those  which  do) 
a base-displacement  method  of  addressing  is  required.  The 
starting  address  of  row  Y is  used  as  a base  and  X is  used  as 
a displacement  in  order  to  find  the  address  of  element 
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I(X,Y)=I  ((address  Y)+X).  This  is  a much  faster  method  of 
indexing  assuming  an  indirect  fetch  is  faster  than  a multiply 
and  two  decrements.  This  method  does  require  some  sort  of 
list  of  the  addresses  of  all  rows,  yielding  for  a regular 
array  N(M+1)  rather  than  NM  storage  requirements. 

That  compression  of  an  image  for  computer  analysis 
differs  from  compression  of  the  image  for  transmission  can 
be  seen  in  the  discussion  above.  Even  though  storage  size  is 
increased  by  base-displacement  addressing,  ease  of  computation 
on  elements  of  the  image  is  increased.  Compaction  schemes 
for  transmission  purposes  are  oriented  toward  minimum  storage 
size  (in  bits,  not  words)  and  minimum  mean  square  error  image 
degradation.  These  goals  may  yield  methods  quite  unsuited 
for  image  analysis  work.  It  is  very  difficult  to  entertain 
the  notion  of  region  growing,  for  example,  in  a differential 
entropy-coded  image.  Some  of  the  methods  of  image  comoression 
for  transmission  such  as  Hadamard  or  Fourier  transforms  are 
useful  in  a feature  extraction  or  enhancement  context  but  the 
overhead  involved  in  converting  to  the  transform  domain  and 
back  may  be  considerable. 

Image  compression  by  run-length  coding  is  an  idea  that 
has  been  around  for  some  time  [3,^,5]  • Its  use,  however,  has 
been  largely  restricted  to  facsimile  transmission  [6,7], 
although  recently  several  computer  graphics  systems  have 
employed  run-length  coding  to  reduce  storage  requirements 
[8,9,10]. 
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B.  Methods 


The  most  natural  form  of  run-length  coding  for  raster- 
type  images  is  horizontal  scan  line  encoding.  The  image 
array  is  scanned  (typically,  left-to-right ) and  an  intensity 
code  and  run-length  are  stored  whenever  the  current  pixel 
intensity  differs  from  the  previously  stored  intensity  code 
by  some  measure.  If,  for  example,  the  measure  is  simply 
the  absolute  difference  between  current  and  previous  inten- 
sities and  a decision  is  made  to  store  whenever  this  measure 
is  greater  than  zero  an  exact  copy  of  the  image  is  stored. 

If  two  words  (intensity  and  run-length)  must  be  stored  at  a 
time,  no  storage  savings  can  result  unless  the  average  of 
the  run-lengths  (RL)  is  greater  than  two.  Thus  the  amount  of 
storage  savings  is  highly  image  and  noise  dependent.  If, 
however,  a length  code  is  stored  only  if  it  is  greater  than 
one  (and  identified  by  a flag  bit)  the  worst  case  performance 
of  the  compaction  scheme  is  exactly  the  same  as  the  unen- 
coded image!  Table  1 lists  RL  and  storage  savings  obtained 
from  several  256  x 256  8-bit  intensity  image  for  different 
decision  thresholds.  Several  theoretical  predictions  of  bit 
storage  savings  (11,12)  for  run-length  coding  have  been  made 
previously.  In  some  applications  environments  the  savings 
may  be  significantly  greater  than  those  indicated  above. 

For  example,  in  an  industrial  environment  a uniform  background 
can  be  provided  behind  the  object  in  view  of  the  television 
camera(s).  This  low  information  background  can  be  encoded 
into  a relatively  small  storage  area. 


Table  1 


•} 

» . 


256  x 256  = 65536  pixels 


Threshold 

# Words 

Savings 

0 

64056 

1.02:1 

4 

37326 

1.76:1 

8 

30539 

2.15:1 

16 

15598 

4.20:1 

32 

8008 

8.18:1 

It  should  be  noted  that  several  schemes  can  be  used 
in  the  storage  decision  process.  Besides  difference  between 
stored  and  current  pixel  values,  the  difference  between 
current  and  previous  pixel  values  | I (X, Y) -I (X-l , 7 ) | can  be 
used.  In  fact,  the  decision  process  is  precisely  that  of 
edge  finding  and  Roberts  [13]  Laplacian,  Hueckel  [14], 
recursive  filtering  [15].,  etc.  methods  may  be  applied. 

The  best  methods  from  a hardware  implementation  stand- 
point are  those  involving  a local  two-dimensional  computation 
or  those  which  can  be  applied  on  an  image  in  raster  scan 
order.  The  choice  of  a two-dimensional  measure  such  as  a 
modified  gradient  yields  the  two  components: 

Dy  = | I* (X, Y-l)  - I (X, Y) | 

Dx  = | I* (X-l, Y)  - I (X,Y) | 

where  I'(i>j)  is  a previously  encoded  intensity  and  I(i,j) 
is  the  current  pixel  Intensity.  In  order  to  achieve  some 
coherency  in  the  vertical  direction  the  procedure  in  Fig.  1 
Is  used.  (THX  and  THY  are  threshold  values).  Essentially, 
whenever  a new  run  is  started,  its  intensity  value  is  that 
of  the  region  above  if  DY  is  small,  and  that  of  the  pixel 
itself  if  Dv  is  large.  This  procedure  is  a mild  bottom-uc 
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region  grower  and  greatly  simplifies  later  edge  location  and 
region  growing  processes.  Again,  from  an  applications  stand- 
point this  method  is  well  suited  to  the  types  of  objects 
potentially  found  in  an  industrial  scene. 

As  discussed  earlier,  data  compaction  for  image  analysis 
is  not  a simple  matter  of  number  of  words  or  bits  saved. 

From  an  access  ease  standpoint  the  run  length  coded  image 
offers  a disadvantage  for  access  to  random  elements.  A list 
of  pointers  to  the  beginnings  of  image  rows  is  kept  separately 
for  indirect  addressing  of  Y.  To  find  the  value  at  a 
particular  X location,  repeated  additions  of  run  lengths  must 
be  made  and  compared  to  the  desired  X location.  If  a variant 
of  the  coding  scheme  is  used  and  final  X location  in  a run 
is  coded  rather  than  run  length,  the  horizontal  access  problem 
is  reduced  to  a search.  The  search  can  be  speeded  considerably 
by  noting  that  line-to-line  coherency  of  the  image  implies 
that  a corresponding  X location  in  an  adjacent  line  can  be 
found  easily.  If  X is  within  the  ith  segment  (run)  of  line  Y, 

X will  be  in  or  close  to  the  i1'*’1  segment  of  lines  Y+l  and  Y-l. 
Thus,  operations  within  a region  located  about  some  X,Y  can 
proceed  with  little  difficulty  in  access.  The  run-length 
scheme  offers  quite  an  advantage  to  computation  involving 
horizontal  elements  3ince  pixel  redundancy  is  indicated  by 
the  run  length.  A major  advantage  of  the  run  length  coded 
data  structure  over  ether  methods  of  reducing  storage  area 
lies  in  the  maintenance  of  positional  accuracy.  That  is,  we 
car.  achieve  a reduction  in  storage  size  by  run  length  coding 
but  maintain  fill  horizontal  and  vertical  resolution  with  only 


6 


\ 


% _ 

j'l 


h 


\ 

I 

r 

*■/ 


i.. 

1 


% 


S 


a minimal  loss  of  intensity  data  and  with  a first  order  edge 
extraction  and  region  growing  procedure  in  the  bargain! 

II . Operations  on  Run  Length  Coded  Images 
A.  Access 

We  shall  consider  a data  structure  representing  an  image 
in  run  length  coded  form  as  follows: 

ILIST  - A list  of  intensity  values  of  each  run  length 
segment 

P.LIST  - A list  of  the  X coordinate  of  the  last  pixel 
In  each  run  length  segment. 

YLIST  - A list  of  pointers  to  the  first  entry  in  P.LIST 
for  each  horizontal  line. 

In  order  to  find  the  intensity  value  at  a given  pixel  location 
X,Y  we  create  a pointer  PTR=YLIST(Y) . If  RLIST(PTR)>X  then 
ILIST (PTR)  contains  the  desired  intensity  value.  Otherwise 
we  increment  PTR  and  check  again,  thus  linearly  searching  a 
section  of  RLIST.  In  order  to  speed  the  search  we  could  adopt 
a binary  search  techniaue  to  some  desired  level. 

n 

As  mentioned  previously  this  is  not  an  efficient  data 
structure  for  random  access.  However,  once  we  know  PTR  for 
some  X,Y  we  can  easily  find  the  intensities  for  horizontally 
nearby  pixels  by  incrementing  or  decrementing  PTR  as  necessary. 

Likewise  vertically  nearby  pixel  intensities  can  be  found  by 
creating  NEWPTR® YPTR ( NEWY ) +OLDPTP- YPTR ( OLDY ) and  incrementing 
or  decrementing  as  necessary.  The  coherency  of  the  picture 
should  ensure  that  this  search  will  be  a short  one.  While 
this  seems  like  a cumbersome  procedure  it  should  be  pointed 
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out  that  every  ILIST  entry  contains  the  value  of  not  just  a 
single  pixel  but  possibly  of  many  pixels  and  that  the  search 
process  gathers  data  about  an  entire  region  of  the  image  and 
not  just  the  few  selected  pixels. 

B . Region  Growing 

A form  of  region  growing  may  be  performed  on  the  run 
length  coded  image.  As  shown  before,  the  original  coding 
scheme  is  a mild  form  of  bottom-up  region  growing.  We  may 
extend  this  procedure  into  the  coded  data  structure.  If,  for 
some  segment  on  line  Y which  ends  at  location  X,  we  examine 
the  intensity  value  of  a corresponding  segment  on  line  Y+l 
and  find  little  difference  between  the  two  we  may  reasonably 
merge  them  into  a single  region  if  and  only  if  there  is  some 
overlap  between  the  two  segments  (see  Fig.  2).  This  procedure 
may  be  carried  out  through  the  image  creating  tags  for  run 
length  segments  assigning  them  to  regions. 

A clean-up  operation  should  then  be  performed  to  merge 
isolated  segments  which  have  no  good  assignment  to  adjacent 
regions.  We  may  then  assign  them  to  the  least  objectionable 
adjacent  region  as  they  are  very  probably  artifacts  of  the  low 
level  coding  merge  and  out  previous  merging  operation. 

C . Boundary  Mapping 

Creating  a boundary  map  from  a run-length  coded  image 
data  structure  is  fairly  straightforward.  RLIST  (original  as 
modified  by  region  growing)  consists  of  all  the  horizontal 
boundary  points.  We  may  find  the  vertical  boundary  points 
by  starting  at  line  Y=2.  If  we  examine  the  difference  between 
a segment  on  line  Y+l  and  line  Y and  find  that  it  exceeds  some 
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selected  threshold  (the  same  operation  as  in  bottom-un  reprion 
growing)  at  any  point (s)  along  the  segment  we  then  add  this 
point(s)  to  the  list  of  vertical  boundary  points  (if  separate 
from  the  horizontal  boundary  point  list). 

If  the  vertical  boundary  point  operation  is  carried  out 
in  an  orderly  top-to-botton,  left-to-right  manner,  the  hori- 
zontal boundary  points  are  obtained  concurrently  (since  we  are 
accessing  segments  for  comparison)  and  the  end  result  is  an 
ordered  list  of  boundary  points  which  may  then  be  used  as 
desired . 

D . Texture 

The  mild  region  growing  and  edge  extraction  performed 
during  the  encoding  process  suggests  a method  of  measuring 
texture  within  some  selected  area  of  an  image. 

Throughout  this  selected  area  we  perform  the  bottom-up 
region  growing  discussed  in  (B)  above  with  a threshold  equal 
to  zero,  omitting  the  isolated  segment  merge  of  (B),  and  then 
average  the  number  of  distinct  regions  found  over  the  selected 
area.  Finally,  we  use  this  average  as  a measure  of  texture. 

Of  course  this  measure  depends  upon  the  thresholds  used  in  the 
run  length  encoding  process,  but  otherwise  does  not  depend 
upon  the  actual  region  values. 

The  region  identification  process  above  is  necessary 
because  while  the  encoding  process  does  take  into  account; 
vertically  adjacent  pixels,  there  is  unfortunately  no  way  to 
indicate  this  explicitly  in  the  encoded  data  structure  without 
developing  a completely  separate  vertical  run  length  encoded 
version  of  the  original  image. 


Edge  Following 


Following  an  edge  through  an  image  is  often  an  important 
step  in  image  analysis.  A procedure  which  follows  a straight 
edge  within  our  run  length  encoded  image  data  structure  will 
be  described. 

During  the  initial  encoding  process,  true  edges  can  very 
often  be  represented  as  several  closely  spaced  edges  as  in 
Figure  3 • The  first  step  in  edge  following  will  be  to  merge 
the  short  run  lengths  which  are  created  by  these  closely 
spaced  edges.  A procedure  for  this  merging  operation  is 
outlined  in  Figure  4.  A similar  procedure  must  be  carried 
out  in  the  vertical  direction.  Unfortunately,  this  is  not 
so  simple  as  in  the  horizontal  direction  but  is  still 
straight  forward . 

Once  these  extraneous  edges  are  reduced  by  the  preceeding 
process  we  proceed  to  follow  the  edge  by  using  a slope  compu- 
tation process.  To  form  our  first  estimate  of  edge  slope 
we  take  the  edge  location  X(Y)  for  row  Y and  the  edge  loca- 
tion X(Y+1)  for  line  Y+l  and  compute  S ( 1=X( Y) -X( Y+l ) . We 
compute  S(2)  in  like  manner  as  (X(Y)-X(Y+2) )/2 . We  use  the 
average  of  S(l)  and  S ( 2 ) as  our  first  estimate  of  the  actual 
edge  slope.  We  use  this  to  preduct  X(Y+2)  in  order  to  speed 
our  access  process  since  the  intensity  of  the  predicted 
X'(Y+2)  identifies  our  prediction  as  being  to  the  left  or 
right  of  the  actual  edge.  We  find  the  actual  X(Y+2)  and  use 
this  to  computer  S C 3 ) and  a new  estimate  of  the  slope  and 
proceed  to  line  Y+3.  We  follow  this  orocedure  until  termination 
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of  the  edge.  Termination  is  detected  by  any  of  several 
conditions : 

(1)  the  image  boundaries  are  exceeded, 

(2)  S(J)  and  S(J+1)  both  differ  from  the  estimated  slope 
by  some  threshold  indicating  the  edge  has  changed 
direction, 

(3)  the  predicted  X'(J)  lies  in  a run  length  with  an  inten- 
sity which  differs  from  that  of  both  regions  lying  on 
either  side  of  the  edge  being  followed  indicating  an 
edge  has  terminated. 

F.  Feature  Extraction 

In  order  to  find  small  regions  of  the  run  length  coded 
which  are  likely  to  contain  features  of  interest  we  follow 
the  ideas  of  Pingle  and  Thomas  [16].  As  a first  step  we 
should  apply  the  multiple  edge  eliminator  described  in  the 
previous  section  on  edge  following.  Then  we  identify  areas 
with  a high  variance  for  further  consideration.  Areas  so 
identified  are  subjected  to  a corner  detector  operation. 

Unlike  most  image  representations,  the  run  length  coded 
data  structure  does  not  lend  itself  most  easily  to  square  or 
rectangular  arrays.  We  will  therefore  consider  a small  area 
of  a fixed  number  of  lines  N but  not  necessarily  a fixed 
number  of  pixels  per  line.  We  shall  decide  that  our  rows 
will  include  at  least  M pixels  and  will  include  complete  run 
lengths,  as  illustrated  in  Figure  5.  Our  areas  will  overlap 
each  other  by  N/4  lines  and  by  one  run  length  per  line.  They 
will  be  arranged  as  in  Figure  6 so  that  at  least  an  MXN 
array  will  be  covered  by  each  area. 


Calculation  of  the  variance  of  an  area  proceeds  quite 
straightforwardly.  We  first  calculate  the  mean  intensity 
I = ( ER( J) I ( J) )/ER( J ) where  P.(J)  is  the  run  length  of  seg- 
ment J and  I(J)  is  the  intensity  of  segment  J. 

The  variance  is  ( ER( J) ( I( J ) -I ) )/ER( J) . If  the  variance 
exceeds  some  given  threshold  we  decide  this  area  is  interest- 
ing enough  to  warrant  further  Investigation,  otherwise  we 
move  onto  the  next  area. 

For  interesting  areas  we  apply  a corner  detector. 

A corner  is  defined  by  the  abrupt  change  in  direction  of  an 
edge  or  by  the  intersection  of  two  or  more  edges.  We  shall 
modify  the  edge  following  technique  described  earlier  because 
of  the  small  size  of  the  area  involved  and  because  of  the 
likelihood  that  a corner  is  within  the  region.  The  measure 
of  slope  we  shall  use  is  X(Y)-X(T+l)  + (XfYH(Y+2))/2.  This 
is  the  initial  estimate  of  slope  referred  to  earlier. 

However,  in  the  present  context  we  shall  not  make  further 
estimates  but  shall  proceed  to  line  Y+2  and  recalculate  the 
slope.  If  the  change  in  slope  exceeds  some  threshold  we 
surmise  that  a corner  or  intersection  is  present . This 
procedure  must  be  carried  out,  of  course,  for  the  edges  of 
each  region  within  the  area  under  consideration. 

G . Cross-correlation  of  Feature  Areas 

Once  we  have  found  two  feature  areas  using  the 
algorithm  of  the  last  section,  we  can  compute  their  cross- 
correlation. Since  the  two  areas  have  the  same  number  of 
pixels  per  line  we  must  describe  the  area  over  which  the 
cross-correlation  will  be  taken  as  in  Figure  7.  The  Xmin 
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and  Xmax  for  a given  line  are  endpoints  of  a segment  in  one 
or  the  other  area  and  we  are  thus  insured  that  we  will 
take  the  cross-correlation  over  at  least  an  MxN  area. 

The  cross-correlation  is  defined  as  EA( I )B ( I)/Z( I ) 

I 

where  A(I)  and  B(I)  are  the  pixel  intensities  of  areas  A and 
B respectively.  Computation  of  this  measure  may  be  speeded 
by  taking  advantage  of  the  run  length  coding  to  coirroute  the 
product  over  segments  rather  than  over  individual  pixels. 
Figure  8 illustrates  a manner  of  doing  so.  The  determination 
of  segment  lengths  is,  in  fact,  not  as  complicated  as  might 
be  supposed  since  each  endpoint  location  is  already  explicit- 
ly stored  in  the  corresponding  RLIST  for  each  area. 

Ill . Implementation 

The  preceeding  methods  developed  for  operations  in  run 
length  coded  Images  will  be  implemented  in  the  Signal  Proces- 
sing Laboratory  during  Spring  1977.  We  expect  this  to  point 
out  problems  in  the  methods  and  to  lead  to  Improved  versions 
of  them.  Basically,  the  ideas  developed  above  are  the  be- 
ginning nucleus  for  the  development  of  a system  for  scene 
analysis  in  encoded  images.  Future  publications  will  deal 
with  the  results  obtained  by  implementation  and  the  modified 
operations  which  result  during  the  learning  process. 


I > 
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A,B,C,D  Intensities  A<B<C<D 
Figure  0 Extraneous  edges  between  regions 
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Run  lengths  i+l  through  J-l  are  merged 
into  R(I ) or  R(J)  depending  on  Intensity 


Figure  4 Extraneous  edge  remova 
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Figure  7 Correlation  length  selection 
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where  5C  represents  an  Intensity  of  C 
for  a run  length  of  5 


IK(X) L(X)  = 2AG  + 2AH  + 5BH  + 5CI  + 1DI  + 1DJ  + 3EJ  + 1PJ 
as  opposed  to 

IK(X)L(X)  = AG+ AG+ AH+ AH+B  H+BH+BK+BH+B  H+CI+CI+CI+CI+CI+ 
DI+DJ+EJ+EJ+EJ+PJ 

Figure  8 Correlation  computation 
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